1. Field of the Invention
The present invention relates to an image processing apparatus and, more specifically, it relates to a structure of a digital image processing apapratus in which video signals from a television camera and the like are filtered in real time using eigher the interlace type of the noninterlace type filtering.
2. BACKGROUND INFORMATION
In the field of image processing, a process called filtering is often used. The filtering is employed to extract a shape having a specific characteristic in an image plane, or for extracting a boundary portion where the brightness in the image plane changes sharply. In the following, the filtering process will be described with reference to the matched filter method as a representative example.
The "matched filter method" is widely wsed in the field of image processing for extracting object regions having a specified intensity distribution and shape which are found here and there in the image plane (or a screen).
In the following , a general process of digital image processing will be described with reference to FIGS. 1A to 1C.
First, an enlarged image of an object to be measured such as erythhrocytes or metal surface grains, is obtained from a television camera through a microscope (FIG. 1A). Image processign is applied to these examples of measurements mainly for the purpose of counting the number of the grains or the like. The troublesome task of counting the grain number has conventionally been done by a skilled operator. However, with the development of the image processing technique, it became possible to automate the task. The processing is carried out in the following manner. First, the analog video signal obtained from a televeision camera is converted into digital signals using an AD (analogue-digital) converter. This step corresponds to the process of dividing the image plane into, for example, 256.times.256 sections or so-called pixels as shwon in FIG. 1B and allocating the intensity value of the image signals to each of the pixels as a digital data. In this step, the more finely the image is segmented, i.e., the larger the number of the pixels is, the higher the resolution of the image becomes. Generally, the pixel number corresponds to 256.times.256 or 512.times.512. Consequently, a certain threshold value is determined for the brightness of the image. It is determined whether a intensity value (or an allocated digital value) of each of the pixels is larger than the threshold value or smaller than the threshold value. The intensity value is then replaced by the signal of "1" or "0" corresponding to the result of the determination (FIG. 1C). This is called the image thresholding. Thereafter, the number of pixels of "1" in the thresholded image data is counted. The method utilizes the fact that the brightness of the object to be counted in the image is brighter (or darker) than the brightness of the background (hatched portions in the figure). However, the brightness of the object to be measured or counted is not so stable as to be separated by a constant threshold value. For example, the brightness of the object widely changes dependent on even a subtle difference in the illuminations. In addition, contaminants of different shapes often exist in the same image plane. Therefore, a method called "matched filter method" is often used for emphasizing and separating the video image of the object having a specified shape and a specified brightness distribution.
FIGS. 2A and 2B illustrate the process of the matched filter method. In the following, the matched filter method will be described with reference to FIGS. 2A and 2B.
As shown in FIG. 2A, image information obtained from a television camera has already been digitized and the respective digital have been a certain brightness value for every pixel. A filter 2 comprising N.times.N (N is an odd-numbered integer) pixels as shown in FIG. 2B is applied to the digitized image 1. The shape of the object which would be extracted is previously represented on the filter 2 as a brightness pattern. It is assumed that F (i,j) denotes the brightness value of the J row, I column position represented in the filter 2, and d (i,j) denotes the pixel data of the j row, i column of the original image 1. The filter 2 is applied to the image 1 in the following manner. Namely, the filter 2 is placed on a certain position of the image plane 1, multiplication of the pixel data d of the image 1 and the pixel data F of the filter 2 is performed for each of the N.times.N pixels overlapping with each other, and addition of all the results of the multiplication is carried out with the sum being the central data of the region where the filter 2 is placed. Thereafter, the filter is moved column by column (in the direction of the thick arrow in FIG. 2A) and the same operation is repeated. This operation is represented by the following equation. ##EQU1## where A is an appropriately selected constant.
After this operation is carried out, only the image regions having the shape matched with the pixel data patern F (I, J) of the filter 2 are emphasized in the image plane.
FIGS. 3A, 3B, 3C and 3D show an actual example employing the matched filter method
In the image plane obtained from a television camera, therer are an object I which should be extracted and a similar object II having a shape similar to the object I (FIG. 3A). When the image on the television screen is digitized, a brightness distribution is obtained which corresponds to the brightness pattern of the objects I and II along the x direction thereof (FIG. 3B). By applying a filter (FIG. 3C) having a simulated brightness pattern with the shape and the brightness distribution of the object I to be extracted from the digitized brightness distribution shown in FIG. 3B, a brightness distribution can be obtained in which the brightness pattern of the object I to be extracted is emphasized (FIG. 3D). By comparing the brightness distribution emphasized by this filter with a certain threshold value, an image thresholding can be carried out in which only the object I to be extraceted, is in fact extracted.
The concept of the filtering has been described using the matched filter method as an example. Other typical filtering methods comprise a unifying process, a boundary line extracting process called Laplacian filter, and so on. These filters are realized by appropriately changing the coefficient values and size of the filter such as shown in FIG. 3C. One example of the unifying process filter is shown in FIg. 4 and one example of the Laplacian filter is shown in FIG. 5, respectively. The manner for these filtering process is the same as that described for the matched filter.
Conventionally, there are two methods for practicing the above described filtering, namely, (1) a method using a computer, and (2) a method using a circuit structured as a dedicated IC.
In the former method, which utilizes a computer, all of the digitized image data are once stored in a memory which is called a frame memory, and the operation represented by the equation (1) is carried out by a program. This method is general-purpose-oriented since the brightness pattern of the filter or the operation following the filtering can be easily selected by the change of the program. However, in this method, the speed of processing depends on the capability of the computer, which is slow in general.
FIG. 6 is a block diagram showing the structure of a dedicated IC which is employed in the second filtering process. In FIG. 6, the dedicated IC comprises a multiplier 5 for multiplying the pixel data FD from the filter and the pixel data ID from the image, an adder 6 which sums up the output of the multiplier 5 and the sum total from a register 7 which stores the sum total output from the adder 6. In the structure of the dedicated IC, first, the pixel data ID from the image and the corresponding pixel data FD from the filter are applied to the multiplier 5, and applied to the adder 6 after the multiplication. The adder 6 receives the output from the multiplier 5 and the sum total till the preceding operation from register 7, sums up thhe both and applies the result to the register 7. The register 7 stores the sum total from thhe adder 6. This operation is repeated N.times.N times which is the number of the pixels of the filter, and thereafter, the sum of the product D.sub.out is outputted from the register 7, thus the filtering of the image is attained.
The filtering method using a circuit with a dedicated IC intends to separate the image filtering process from a computer to further increase the speed of processing. In general, the commercially available image processing device employ this second method wherein the data are continuously stored in the register 7 until the completion of the N.times.N operations. Therefore, the image data on the image plane should be once stored in the frame memory, so that the speed of operation is not so much improved as to realize a processing of the same speed as the data process rate on the television image (processing of one image plane in 1/30 second).
The disadvantages of the above described conventional image processing systems will be described with reference to FIG. 7. Referring to FIG. 7, the conventional image processing system comprises a television camera 90 which picks up the object and generates an image information (video signal) corresponding to the object, A/D converter 91 which samples an analogue video signal .THETA. from the television cammera 90 with a prescribed frequency and quantizes the sampled signal to generate a digital signal .PHI., a frame memory 92 for storing the digital signal .PHI. from the A/D converter 91 for one image plane (one frame), and an image processing apparatus for carying out image processing such as a prescribed filtering on the digital image signal .PHI. from the frame memory 92.
The number of pixels constituting one image plane is determined by the sampling frequency in the A/D converter 91. In general, the sampling frequency is determined so as to divide one image plane into 256.times.256 or 512.times.512 pixel numbers.
The frame memory 92 stores information of color or contrast for every pixel. After the pixel data for one image plane comprising approximately 256.times.256 or 512.times.512 pixels are stored in the frame memory 92, the pixel data are read successively by the image processing apparatus 93 in the order of the writing to the memory and a predetermined process is carried out.
In order to effect real time processing in which the video signals .THETA. obtained from the television camera 90 is processed at the same speed, the processing faculty corresponding to the frame frequency of the television image plane (or screen), that is, 1/30 second is needed. Namely, the image processing must be carried out 30 times per second.
As for the image processing, the real time processing has become possible to same extent by virtue of the development of the dedicated LSI or of the improvements in circuit-confituration. However, in the conventional structure of the apparatus, all the pixel data for one image plane is once written in the frame memory 92 and then the image processing apparatus 93 reads the pixel data from the frame memory 92. Therefore, writing and reading of the pixel data for one image plane to and from the frame memory 92 must be carried out, so that the speed of processing is decreased to 1/2, whereby real time processing is no longer possible.
On the other hand, the real time image processing has been strongly desired due to the recent trend of speed up in factory automation and so on.